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W.  H.  Fleming  continued  his  work  on  diffusion  processes  depend- 
ing on  small  parameters  and  on  optimal  stochastic  control.  In 
joint  work  with  C-P  Tsai,  differential  games  techniques  were 
used  to  study  minimum  exit  probabilities  for  controlled  dif- 
fusions, when  the  noise  entering  the  system  has  low  intensity. 

The  results  are  formally  similar  to  the  asymptotic  estimates 
of  Ventsel -Freidlin , but  the  method  of  proof  is  quite  different. 
It  depends  heavily  on  ideas  from  control  theory  and  game  theory. 

A separate  line  of  work,  by  Fleming  and  Viot,  concerns  measure- 
valued diffusion  processes.  The  particular  class  of  distributed- 
parameter  Markov  stochastic  processes  they  treated  arose  from  a 
population  model.  However,  the  form  of  the  generators  of  these 
Markov  processes  implies  a certain  useful  structure  to  the 
partial  differential  equations  for  means  and  covariances  of 
population  numbers  in  different  regions.  A similar  moment 
structure  appears  in  interesting  physical  applications,  for 
instance,  in  the  study  of  wave  propagation  with  a continuum  of 
modes  in  a random  medium.  The  Fleming-Viot  method  turns  out 
to  be  of  interest  for  such  problems  as  well. 


Papers  published 

W.H.  Fleming,  "Exit  probabilities  and  optimal  stochastic  control", 
Applied  Math,  and  Optimization,  4(1978),  p.329-346. 

W.H.  Fleming  and  C-P  Tsai,  Minimum  exit  probabilities  and  dif- 
ferential games.  Differential  Games  and  Control 
Theory  III,  Eds.  P-T.  Liu  and  E.  Roxin  (Proc. 
Kingston  Conference,  June  1978),  Lecture  Notes 
in  Pure  and  Applied  Math  4_4,  1979,  Marcel  Dekker, 
pp.  67-76. 
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W.H.  Fleming  and  M.  Viot,  Some  measure- valued  population 

processes,  Stochastic  Analysis,  Eds.  A. 

Friedman  and  M.  Pinsky  (Proc.  Northwestern 
Conference,  April  1978),  Academic  Press, 
pp.  97-108. 

J.  Ruzicka  visited  Brown  during  April  - June  1978.  His  work 
dealt  both  with  basic  theoretical  questions  in  stochastic  control 
and  with  an  aerospace  application  of  the  separation  principle. 

The  latter  work  is  summarized  in  the  report  "Research  in  problems 
associated  with  flight  control  in  turbulence". 

R.  J.  Elliott  visited  Brown  during  May,  June  1978.  His  work 
concerned  optimality  conditions  for  stochastic  systems  under 
partial  observations.  A paper  by  Elliott  and  P.P.  Varaiya 
"A  sufficient  condition  for  the  optimal  control  of  a partially 
observed  stochastic  system"  was  written  during  this  period. 

This  is  part  of  a substantial  body  of  work  by  Professor  Elliott 
in  this  area,  done  recently. 

The  work  [1]  was  completed  by  Kushner , dealing  with  the 
approximation  and  computational  problem  for  the  average  cost 
per  unit  time  problem  for  a diffusion  model.  In  our  past  work 
[2] , we  developed  a fairly  comprehensive  approach  to  the  com- 
putational problem  for  optimal  control  problems  on  diffusion  and 
jump-diffusion  models.  Ref.  [1]  is  a non-obvious  extension  of 


the  ideas  in  [2]  to  the  practically  important  average  cost  per 
unit  time  problem.  Difficulties  are  caused  by  the  fact  that 
the  average  cost  problem  involves  control  over  an  infinite  time 
interval;  in  particular,  little  is  known  concerning  the  relations 
between  existence  of  invariant  measures  of  the  approximating 
chains  and  the  diffusion.  This  is  an  important  question,  since 
it  is  these  measures  which  determine  the  costs,  and  even  if  the 
diffusion  is  nicely  behaved,  the  approximating  chain  may  not  be, 
and  vice  versa. 

Stochastic  approximation  methods  for  constrained  and  uncon- 
strained systems . 

Kushner  has  published  the  monograph  [3] , which  covers  an 
enormous  amount  of  material  and  new  ideas  and  techniques  for  many 
constrained  and  unconstrained  problems  of  the  stochastic  approxi- 
mation type.  It  uses  much  better  and  more  realistic  conditions 
and  easier  and  more  intuitive  methods,  and  gets  better  results 
than  the  more  classical  works  in  the  area.  Furthermore,  it 
gets  better  rates  of  convergence. 

Many  problems  in  adaptive  control  and  identification  theory 
involve  recursive  equations  such  as  (1) 


X , = X + a H(X  ,E  ) , 
n+1  n n v n ’^n'  ’ 


(1) 


where  {£n>  is  a noise  sequence,  an 


positive  and  real,  and 
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IaR  = but  where  II  is  not  additive  in  the  noise  £.  It  is, 
of  course,  a stochastic  approximation  type  of  algorithm  and  con- 
vergence is  treated  in  [3],  for  the  case  where  an  -*■  0.  But 
no  work  had  been  done  on  the  rate  of  convergence.  It  turns  out 
[6]  that  we  can  get  results  as  extensive  as  those  for  the  case 
where  II  is  additive  in  £.  This  is  particularly  important 
since  it  is  usually  by  a comparison  of  rates  of  convergence 
that  one  gets  a useful  comparison  between  competing  algorithms. 
The  conditions  in  [6]  are  quite  reasonable  for  the  types  of 
applications  that  we  have  in  mind. 
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